Image processing apparatus, method, and program using depression time input

ABSTRACT

An image processing apparatus which includes an extension width determination unit for determining an extension width based on a depression time of the cursor at a reference position on an image display unit where a releasing operation of the cursor was performed, which is a time during which the cursor had been kept depressed until the releasing operation was performed, and an ornament piece arrangement unit for arranging the plurality of ornament pieces at positions radially extended away from the reference position with the extension width determined by the extension width determination unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/471,596, filed May 26, 2009 and entitled “Image Processing Apparatus,Method, and Program Using Depression Time Input,” which claims priorityto Japanese Patent Application Ser. No. 2008-136592, filed May 26, 2008and entitled “Image Processor and Processing Method, and Program,” whichare hereby incorporated by reference herein.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus andmethod for arranging a plurality of ornament pieces in a target areadisplayed on an image display device based on a cursor operation. Theinvention also relates to a computer readable recording medium on whichis recorded a program for causing a computer to perform the imageprocessing method.

2. Description of the Related Art

Recently, it has been practiced for pleasure to decorate photo imageswith various types of graphic ornaments, such as flower, heart shape,and the like, and a method in which a photo image to be decorated isdisplay on a display device and a desired ornament piece is arranged ata position directly specified by the user by clicking a cursor or thelike is known.

As shown in FIG. 14, Japanese Unexamined Patent Publication No.2000-022929 (Patent Document 1) proposes a method for automaticallyarranging ornament pieces at random positions of the background area ofphoto image 91 other than face area 92.

Further, U.S. Patent Application Publication No. 20040125423 (PatentDocument 2) proposes a method for automatically arranging an ornamentpiece (glasses, false whiskers, or the like) on a person's face area ofa photo image by enlarging or reducing the ornament piece according tothe size of the face area and positioning the ornament piece so as tomatch with the image of the face area.

The method in which an ornament piece is arranged at a positionspecified directly by the user, however, requires a position specifyingoperation each time an ornament piece is arranged. Therefore, the methodhas a problem that it requires more time for a larger number of ornamentpieces to be arranged. Further, where the user is not accustomed to thepositioning for arranging an ornament piece, it is often the case thatthe arrangement results in unnatural.

Further, the method described in Patent Document 1 in which ornamentpieces are arranged automatically at random positions may eliminate thetime and effort of the user, but may not reflect user's intention in thearrangement of ornament pieces.

Still further, the method described in Patent Document 2 in which theappropriate arrangement position of an ornament piece is uniquelydetermined has a problem that the variation of decorative effectsprovided by the ornament piece is limited.

In view of the circumstances described above, it is an object of thepresent invention to provide an image processing method and apparatuscapable of arranging a plurality of ornament pieces in a manner thatreflects user's intention by a simple cursor operation. It is a furtherobject of the present invention to provide a computer readable recordingmedium on which is recorded a program for causing a computer to performthe image processing method.

SUMMARY OF THE INVENTION

An image processing apparatus of the present invention is an apparatus,including:

an extension width determination unit for determining an extension widthbased on a depression time of the cursor at a reference position on theimage display unit where a releasing operation of the cursor wasperformed, the depression time being a time during which the cursor hadbeen kept depressed until the releasing operation was performed; and

an ornament piece arrangement unit for arranging the plurality ofornament pieces at positions radially extended away from the referenceposition with the extension width determined by the extension widthdetermination unit.

In the apparatus described above, the extension width determination unitmay be a unit that determines the extension width such that the longerthe depression time the greater the extension width.

Further, the ornament piece arrangement unit may be a unit that arrangesthe plurality of ornament pieces along an arrangement line having apredetermined shape and being extended away from the reference positionwith the extension width determined by the extension width determinationunit.

Here, the predetermined shape may be any of circular shape, ellipsoidalshape, polygonal shape, star shape, and heart shape.

The apparatus described above may further include a setting unit forsetting at least one of the number, size, and color of ornament piecesto be arranged by the ornament piece arrangement unit.

The setting unit may be a unit that performs the setting based on adepression time of the cursor on a selection button used to select thetype of the ornament pieces to be arranged displayed on the imagedisplay unit.

Further, the apparatus described above may be an apparatus configuredsuch that the depressing operation of the cursor on the selection buttonand the releasing operation of the cursor at the reference position areperformed in one continuous action.

Still further, in the apparatus described above, the target area may bean area in which a photo image is present, the apparatus may furtherinclude a detection unit for detecting an area representing a mainsubject in the photo image, and the ornament arrangement unit may be aunit that arranges the ornament pieces in the area detected by thedetection unit and the other area in different arrangement forms.

An image processing method of the present invention is a method forarranging a plurality of ornament pieces in a target area displayed on adisplay unit based on an operation of a cursor, the method including thesteps of:

determining an extension width based on a depression time of the cursorat a reference position on the image display unit where a releasingoperation of the cursor was performed, the depression time being a timeduring which the cursor had been kept depressed until the releasingoperation was performed; and

arranging the plurality of ornament pieces at positions radiallyextended away from the reference position with the extension widthdetermined by the extension width determination unit.

A computer readable recording medium of the present invention is amedium on which is recorded a program for causing a computer to performthe method described above.

In the image processing method, apparatus, and program, the referenceposition is not limited to a position in the target area and maybe anyposition on the display unit. It is necessary, however, that at leastsome of the positions radially extended away from the reference positionwith the extension width determined based on the depression time of thecursor at the reference position are present in the target area forarranging the ornament pieces in the target area.

The term “target area” as used herein broadly refers to an area whereornament pieces are intended to be arranged, and includes an area wherecontents, such as various types of images, graphics, character strings,or the like are present, as well as an area where no contents arepresent.

The term “cursor” as used herein refers to an operation position on ascreen and not necessarily limited to a dingbat displayed on the screen.Accordingly, the term “operation of a cursor” as used herein refers toriot only the operation of the dingbat on the screen for operating thecursor using an input device for specifying an input position orcoordinates on the display screen, such as mouse, infrared or wirelessremote control pointing device, or the like, but also a touch paneloperation with a finger in which nothing appears on the screen.

Here, for example, when the input device for operating the cursor is amouse, the terms “depressing operation of the cursor, keeping the cursordepressed, moving the cursor while depressed (dragging operation), andreleasing operation of the cursor” as used herein refer to thedepressing operation of the mouse button, keeping the mouse buttondepressed, moving the cursor with the mouse button being depressed, andreleasing operation of the cursor respectively. If it is a touch panel,the touching operation to the touch panel with a finger, keeping thefinger touched to the panel, moving the touched finger on the panelsurface, and releasing operation of the finger from the panel.

Further, the term “depressing operation of the cursor and the releasingoperation of the cursor are performed in one continuous action” as usedherein refers to that the cursor is kept depressed between the twooperations.

According to the image processing method, apparatus, and program of thepresent invention, when arranging a plurality of ornament pieces in atarget area displayed on a display unit based on an operation of acursor, an extension width is determined based on a depression time ofthe cursor at a reference position on the image display unit where areleasing operation of the cursor was performed, the depression timebeing a time during which the cursor had been kept depressed until thereleasing operation was performed, and the plurality of ornament piecesare arranged at positions radially extended away from the referenceposition with the extension width determined by the extension widthdetermination unit. This allows the plurality of ornament pieces to bearranged in a manner that reflects user's intention by a simple cursoroperation.

Further, when at least one of the number, size, and color of ornamentpieces to be arranged is set based on the duration of keeping the cursordepressed on a selection button used to select the type of the ornamentpieces displayed on the image display unit, the user may set the numberof ornament pieces to be arranged or the like by an intuitive operationof the cursor.

Still further, when a depressing operation of a cursor on a selectionbutton and a releasing operation of the cursor at a reference positionare performed in one continuous action, ornament pieces may be arrangedwith a less number of operations.

Further, where the target area is an area in which a photo image ispresent, an area representing a main subject in the photo image isdetected, and ornament pieces are arranged in the detected area and theother area in different arrangement forms, then, for example, manyornament pieces may be arranged in an area other than a face area, i.e.,the main subject area, of a portrait image, whereby the face image maybe emphasized.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram of an image processingworkstation according to an embodiment of the present invention.

FIG. 2 is a block diagram of an ornament piece arrangement function ofthe image processing unit shown in FIG. 1.

FIG. 3 illustrates an example screen displayed on the image display unitshown in FIG. 1.

FIG. 4 is a drawing for explaining a selection operation for an ornamentpiece and a setting operation for the number of pieces to be arranged.

FIG. 5 illustrates an example auxiliary line displayed by the auxiliaryline display control unit shown in FIG. 2.

FIG. 6 is a drawing for explaining an arrangement operation of ornamentpieces along an arrangement line.

FIG. 7 is a drawing for explaining an arrangement operation of ornamentpieces along an arrangement line.

FIG. 8 illustrates an example arrangement in which ornament pieces arearranged along an arrangement line.

FIG. 9 illustrates an example arrangement in which ornament pieces arearranged so as not to overlap with a face area.

FIG. 10 illustrates another example arrangement in which ornament piecesare arranged so as not to overlap with a face area.

FIG. 11 illustrates still another example arrangement in which ornamentpieces are arranged so as not to overlap with a face area.

FIG. 12 is a drawing for explaining an arrangement operation of ornamentpieces along an ellipsoidal arrangement line.

FIG. 13 illustrates an example arrangement in which ornament pieces arearranged along an ellipsoidal arrangement line.

FIG. 14 illustrates an example ornament piece arrangement performed by aconventional method.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an exemplary embodiment of the present invention will bedescribed with reference to the accompanying drawings.

FIG. 1 is a block diagram of an image processing workstationillustrating the schematic configuration thereof. The image processingworkstation is a computer for displaying an inputted image performingimage processing in response to an instruction from the user, anddisplaying a processed image. As shown in FIG. 1, the workstationincludes image input unit 10 for receiving a digital image, user inputunit 20 for receiving a user instruction, image processing unit 30 forperforming image processing on the digital image based on input receivedby user input unit 20, image display unit 40, such as a display or thelike, and display control unit 50 for causing image display unit 40 todisplay an inputted image, processed image, and the like.

User input unit 20 includes a mouse, a touch panel, a remote controlpointing device, or the like for specifying an input position orcoordinates on a display screen of image display unit 40. The user maycause the computer to perform image processing to be described later byoperating a cursor on a display screen using user input unit 20.

FIG. 2 is a block diagram of a portion of image processing unit 30related to the function of arranging ornament pieces. Image processingunit 30 is a unit for arranging ornament pieces, for example, on targetarea 74 (area in which a photo image is present) displayed on displayscreen 60 shown in FIG. 3 based on a cursor operation. It includesornament piece selection unit 31, intensity setting unit 32, extensionwidth determination unit 34, auxiliary line display control unit 35,ornament piece arrangement unit 36, and detection unit 37.

Display screen 60 shown in FIG. 3 includes ornament piece selectionbuttons, image processing area 61 for displaying target area 74, andmenu display area 63 for displaying selection menu items used forselecting the type of image processing performed by image processingunit 30.

The configuration of the image processing unit described above isrealized by executing an image processing program, read in an auxiliarystorage device, on the image processing workstation. Here, the imageprocessing program is stored in information recording medium, such asCD-ROM or the like, or distributed through a network, such as theInternet, and installed on the computer.

Ornament piece selection unit 31 is a unit for selecting, when, forexample, a cursor is depressed on one of selection buttons 71, 72, 73displayed on image processing area 61 shown in FIG. 3, the correspondingtype of ornament piece.

Intensity setting unit 32 is a unit for setting the number, size, color,and the like of ornament pieces using predetermined default values or inresponse to user input. For example, the number of ornament pieces isdetermined according to the number of depressing operations of cursor ona selection button, duration of operation maintained during the cursoris depressed, depression force when the cursor is depressed using apressure sensitive touch panel.

As shown in FIG. 4, an animation may be displayed in which a longerdepression time of cursor Cs on a selection button causes more heartshaped ornament pieces to be displayed and more largely.

When arranging ornament pieces of various sizes and colors, the varietyof the ornament pieces to be arranged may be controlled by setting thenumber of colors, number of sizes, and the like.

Further, an arrangement may be adopted in which a setting value of eachof a plurality of setting items is predefined for a plurality of stagesand a setting value set corresponding to the stage selected according touser input is used. For example, the definition is made such that thenumber of ornament pieces is set to five in stage 1, in which two ofthem in red, the rest of three in white, and two of them in large size,another two of them in medium size, and the rest of one in small size,and the number of ornament pieces is set to seven in stage 2, in whichtwo of them in red, three others in white, and the rest of two in blue,and three of them in large size, two others in medium size, and the restof two in small size.

Extension width determination unit 34 determines the extension width “d”of an arrangement line “E” used for arranging ornament pieces inornament piece arrangement unit 36 to be described later based on adepression time of cursor Cs at a reference position “P” on displayscreen 60 where a releasing operation of cursor Cs was performed, whichis a time during which cursor Cs had been kept depressed until thereleasing operation was performed.

More specifically, for example, the extension width “d” of thearrangement line “E” from the reference position “P” is determined suchthat the longer the elapsed time from the start of depressing operationof cursor Cs to the releasing operation of cursor Cs, the greater theextension width. Where the arrangement line is a circle, the extensionwidth “d” may be the radius of the circle, and if the arrangement lineis an ellipsoid, the extension width “d” may be the radius in a longaxis direction.

Here, the arrangement line may have various shapes, such as circularshape, ellipsoidal shape, polygonal shape, star shape, hear shape,S-shape, and the like. Further, the arrangement line is not limited toclosed shapes.

Extension width determination unit 34 is not limited to a unit thatdetermines the extension width “d” of the arrangement line “E” such thatthe longer the depression time of cursor Cs at a reference position “P”where a releasing operation of cursor Cs was performed, which is a timeduring which cursor Cs had been kept depressed until the releasingoperation was performed, the greater the extension width “d”. It may bea unit that determines the extension width “d” such that the extensionwidth “d” is gradually reduced from a predetermined reference value asthe depression time of cursor Cs is increased.

Support line display control unit 35 is a unit for displaying at leasteither one of an arrangement line “E” and a segment “D” representing theextension width “d” of the arrangement line “E” from a referenceposition “P” determined at each time point where cursor Cs is releasedat the reference position “P” during the time from the start ofdepressing operation of cursor Cs to the releasing operation of cursorCs at the reference position “P”, as shown in FIG. 5. As describedabove, the arrangement line “E” is defined such that the longer thedepression time of cursor Cs at the reference position “P”, the greaterthe extension width “d”, so that the auxiliary line (arrangement line“E” and/or segment “D”) displayed by auxiliary line display control unit35 is extended farther away from the reference position “P” as thedepression time of cursor Cs at the reference position “P” is extended.This allows the user to predict the relation between the depression timeof cursor Cs and the extent where ornament pieces are arranged, so thatthe user may arrange the ornament pieces in a desired area by referringto an auxiliary line displayed by auxiliary line display control unit 35while maintaining cursor Cs in depressed state at the reference position“P” and releasing cursor Cs at a time point when the auxiliary linebecomes a desired arrangement line “E” (or the extension width “d”thereof).

Ornament piece arrangement unit 36 is a unit for arranging ornamentpieces along the arrangement line “E” having an extension width “d” fromthe reference position “P” determined by extension width determinationunit 34. For example, the ornament piece arrangement unit 36 setsequally spaced reference points on a circular arrangement line “E” witha radius of extension width “d” centered on a reference position, thenwith respect to each of the reference points, sets an equilateraltriangle having a random size with its gravity point at the referenceposition, and arranges an ornament piece at each corner of theequilateral triangle, as shown in FIGS. 6 and 7. This produces an imagewith ornament pieces arranged in the manner shown in FIG. 8.

Ornament piece arrangement unit 36 further has a function for arrangingornament pieces in a main subject area and the other area detected by aconventional object detection method in detection unit 37 in differentarrangement forms. For example, where the face area in the portraitimage shown in FIG. 8 is detected as the main subject, it is possible todelete ornament pieces arranged on the face area as shown in FIG. 9 orotherwise to rotate or reduce the triangles for arranging the ornamentpieces so as not to be arranged on the face area, as shown in FIG. 10.

Further, by treating the detected main subject area as a different layerfrom that of the other area, for example, the ornament pieces may bearranged at the back of the main subject (here, the entirety of theportrait area) and in front of the other area, as shown in FIG. 11.

In the arrangement of ornament pieces described above, the number ofreference points and type of polygonal shape (triangular shape,rectangular shape, combination thereof, and the like) may be determinedbased on the number of ornament pieces set by intensity setting unit 32,and the spacing between the reference points may be determined accordingto the size of the photo image.

The size of the polygonal shape set at each reference point may berandom. Alternatively, the size may be preset so as to be graduallyincreased or decreased along the arrangement line.

Further, each polygonal shape may be a shape that has the correspondingreference point at its rough center or at a position away from thecenter, or a shape rotated at an arbitral angle with respect to apredetermined reference axis having the reference point at its centerposition.

FIGS. 12 and 13 illustrate an example case in which ornament pieces arearranged along an ellipsoidal arrangement line “E”.

When arranging ornament pieces using the image processing workstation ofthe present invention configured in the manner described above, the usermay arrange, for example, a desired number of ornament pieces in adesired area of a target image by a series of intuitive cursordepressing and releasing operations, that is, by depressing the cursoron a selection button corresponding to the desired type of ornamentpieces to be arranged displayed on a display screen, keeping the cursoron the selection button while depressing the cursor until the number,size, and color are set, then moving the cursor to a position that canbe the reference position of an arrangement line on which the ornamentpieces are arranged while keep depressing the cursor when a desiredsetting is obtained, and referring to an auxiliary line displayed byauxiliary line display control unit 35 while still keep depressing thecursor at the reference position, and releasing the cursor at a timepoint when the auxiliary line becomes a desired arrangement line “E” (orthe extension width “d” thereof).

In the example operation, the description has been made of a case inwhich a cursor depressing operation on a selection button for ornamentpiece selection and the like, and a cursor releasing operation forinstructing the ornament piece arrangement are performed in onecontinuous action. Alternatively, an arrangement may be adopted in whichafter the cursor depressing operation is performed on a selection buttonfor ornament piece selection, and the Like, the cursor is depressedagain and released for arranging the ornament pieces.

The ornament piece is not limited to a graphical shape, such as flower,heart shape, or the like, and it may be, for example, a reduced ordinaryimage or an object image extracted from an image.

1-20. (canceled)
 21. A computer implemented method comprising:detecting, by a computer system, user engagement with an input unit;determining, by the computer system, an extension width based on a timeof user engagement with the input unit; arranging, by the computersystem, ornament pieces on an image at different positions radiallyextended away from a reference position based on the extension width;based on user engagement with the input unit and at least a first of aplurality of selection buttons, selecting, by the computer system, ashape-type of the ornament pieces, each of the plurality of selectionbuttons indicating an available different shape-type of the ornamentpieces; and based on the time of user engagement with the input unit andthe at least a first of a plurality of selection buttons, setting, bythe computer system, at least one of a number of the ornament pieces, asize the ornament pieces, and a color of the ornament pieces.
 22. Themethod of claim 21, wherein the determining an extension width includessetting the extension width to be proportional to the time of userengagement with the at least a first of a plurality of selectionbuttons.
 23. The method of claim 21, wherein the arranging ornamentpieces includes locating the ornament pieces at a distance from thereference position by the extension width.
 24. The method of claim 21,wherein the arranging ornament pieces includes locating the ornamentpieces along an arrangement line having a predetermined shape.
 25. Themethod of claim 24, wherein the predetermined shape is at least one of acircular shape, an ellipsoidal shape, a polygonal shape, a star shape,and a heart shape.
 26. The method of claim 24, wherein the predeterminedshape is a polygon and the ornament pieces are located at angles of thepolygon.
 27. The method of claim 24, further comprising rotating thepredetermined shape.
 28. The method of claim 24, further comprisingreducing the predetermined shape.
 29. The method of claim 21, whereinthe input unit detects user contact.
 30. The method of claim 29, whereinuser engagement with the input unit and the at least a first of aplurality of selection buttons includes user contact with the input unitand the at least a first of a plurality of selection buttons.
 31. Themethod of claim 30, wherein the time of user engagement with the inputunit and the at least a first of a plurality of selection buttons is atime of user contact with the input unit and the at least a first of aplurality of selection buttons.
 32. The method of claim 21, furthercomprising detecting a first area representing a main subject of theimage and a second area of the image.
 33. The method of claim 32,wherein the arranging ornament pieces includes arranging the ornamentpieces in the first area and the second area in different arrangementforms.
 34. The method of claim 32, wherein the arranging ornament piecesincludes locating the ornament pieces only in the second area.
 35. Themethod of claim 32, wherein the first area includes at least one face.36. A non-transitory computer-readable medium having stored thereon aset of instructions, which when executed by a computer, perform amethod, comprising: detecting user engagement with an input unit;determining an extension width based on a time of user engagement withthe input unit; arranging ornament pieces on an image at differentpositions radially extended away from a reference position based on theextension width; based on user engagement with the input unit and atleast a first of a plurality of selection buttons, selecting ashape-type of the ornament pieces, each of the plurality of selectionbuttons indicating an available different shape-type of the ornamentpieces; and based on the time of user engagement with the input unit andthe at least a first of a plurality of selection buttons, setting atleast one of a number of the ornament pieces, a size the ornamentpieces, and a color of the ornament pieces.
 37. The computer-readablemedium of claim 36, wherein the determining an extension width includessetting the extension width to be proportional to the time of userengagement with the at least a first of a plurality of selectionbuttons.
 38. The computer-readable medium of claim 36, wherein thearranging ornament pieces includes locating the ornament pieces at adistance from the reference position by the extension width.
 39. Thesystem comprising: at least one processor, and a memory storinginstructions configured to instruct the at least one processor toperform: detecting user engagement with an input unit; determining anextension width based on a time of user engagement with the input unit;arranging ornament pieces on an image at different positions radiallyextended away from a reference position based on the extension width;based on user engagement with the input unit and at least a first of aplurality of selection buttons, selecting a shape-type of the ornamentpieces, each of the plurality of selection buttons indicating anavailable different shape-type of the ornament pieces; and based on thetime of user engagement with the input unit and the at least a first ofa plurality of selection buttons, setting at least one of a number ofthe ornament pieces, a size the ornament pieces, and a color of theornament pieces.
 40. The system of claim 39, wherein the determining anextension width includes setting the extension width to be proportionalto the time of user engagement with the at least a first of a pluralityof selection buttons.